An Automatic Segmentation Algorithm Based on Chinese Phoneme

The primary task of Chinese language processing is to establish efficient and accurate segmentation strategy. With the Chinese’s characteristics of been idea-phonetic language, the paper advances an automatic segmentation algorithm that is based on Chinese phoneme to realize disambiguation. First, the candidate tag set, which consists of ambiguous phrases that result from Chinese polyphones, is built up, and every possible segmentation result of each phrase compose the segmentation tag set, then, the calculation of posterior probability is transformed into solving optimization problem, and with genetic algorithm to get the optimal solution, furthermore, the approach also resolve the sparse data problem in HMM. The experiment shows that with this method to solve the ambiguity caused by polyphones is practicable and has a good effect.